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ABSTRACT 

Second language tests claiming to assess 
communicative competence are widespread, despite the vague nature of 
the construct. Sociol inguistic or inter cul tural competence is 
gradually gaining attention in the classroom, but testing has not 
kept pace, partly because of difficulty in defining the related 
skills. An opinion is that speech act theory and politeness theory, 
which focus on social aspects of communicative exchange while 
remaining true to pragmatic principles, can inform development of 
appropriate tests. Role playing and simulation are authentic means of 
testing overall communicative competence. The Discourse Completion 
Test (DCT) , designed to elicit responses to problematic, contextually 
specific prompts as participants role play their responses, is 
currently under analysis as both a research tool and a testing 
option, as an alternative to the oral proficiency interview. The DCT 
capitalizes on the benefits of role playing and applies them to 
assessment. However, contextual aspects of DCTs must be better 
developed to convey more about interlocutors' relationship (status, 
positional identities). While the communicative objectives of the DCT 
are in line with pragmatic principles, the method needs to better 
reflect clear pragmatic criteria. Contains 42 references. (I'lSE) 
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Assessing L2 Sociolinguistic Competence: 

In Search of Support from P Tigmatic Theories 



With a heavy emphasis on communicative approaches to 
langiiage teaching, many culturally-specific norms and rules 
of speech behavior in English have gradually found their 
way into second language classrooms. In spite of the 
overwhelming acceptance of erq)hasizing cross-cultural 
awareness in the teaching-learning process, it has remained 
an imcertain area in language testing. Such measurement 
devices have proven more or less dependable at evaluating 
the necessary grammatical competence of most nonnative 
speakers of English, but have yet to demonstrate sensitivity 
to sociocultural factors. Intercultural pragmatic success or 
"failure" (Thomas 1983) needs to be incorporated into the 
assessment of ESL/EFL students as well. Authentic means 
for pursuing this objective warrant the use of roleplays and 
simulations for testing overall communicative competence, 
which by definition, includes a sociolinguistic component. 
However, face validity alone is not satisfactory; construct 
validity should be the primary concern in test development. 
Accordingly, we must continue to work out the specifics of 
the psychological trait we wish to assess by examining the 
theoretical frameworks in use in this area-Speech Act theory 
and Politeness Theory. Their contributioiiS and short- 
comings to L2 testing in terms of measuring sociolinguistic 
abilities are discussed from the context-oriented perspective 
of semiotics. 



Robin D. Zuskin 
University of New Mexico 
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INTRODUCTION 



Tests claiming to assess communicative competence are rampant despite the 
vague nature of this super construct. Along with the gradual acceptance of a 
componential framework accounting for successful and problematic talk (Canale 
& Swain 1980), sociolinguistic competence*-otherwise known as sociocultural, 
intercultural, or pragmatic competence— is gradually gaining attention. Classroom 
teaching materials have begun to reflect this trend toward discussing situation 
perception skills (see Wolfson 1989) of register variation, status, social distance, 
and face-related issues while, ironically, most L2 assessment procedures remain 
aloof. 

Part of the reason behind this procrastination stems from the task of measuring 
cognitive skills: in order to test a psychological trait, we must be able to clearly 
define and explain it. For test developers, construct validity must remain a 
priority. Before they can assess whether or not second language (L2) students 
know "when to speak, when not, and as to what to talk about, with whom, where 
and in what manner" (Hymes’ 1972 original definition of communicative 
competence), they had better be clear on how to view it in the grander scheme of 
things. Members of the applied linguistics community need to investigate if a 
proliferation of pragmatic theories has gotten out of hand (Zuengler 1992). It is 
my contention that theoretical reorganization is in order so that we avoid 
redundancy and overlap: a broader pragmatic framework which incorporates 
some of the sociocultural aspects necessary to explain and predict human 
communication. This umbrella perspective is best imderstood visually (Fig, 1): 
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Figure 1: The pragmatic umbrella 
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As this diagram illustrates, several theoretical positions contribute to our 
understanding of the message exchange process in face-to-face communication. 
The problem is that we have a tendency to limit or confme the power of our 
theory in order to explain a specific aspect of this process. In the area of 
assessing first language sociolinguistic transfer— the carry-over of sociocultural 
norms and implicit behavioral guidelines during second language acquisition and 
interlanguage processes— a near blind acceptance of two prototypical pragmatic 
theories has emerged in the literature: Speech Act Theory and Politeness Theory. 
When used for L2 assessment objectives, these frameworks determine testing 
criteria. Sensitivity to illocutionary force of various speech acts or to appropriate 
politeness norms during facework have frequently served in this role. Both 
theories focus on social aspects of communicative exchanges while remaining true 
to pragmatic principles. However, neither frameworie in isolation can handle the 
entire explanatory process; along with other problems, an understanding of the 
interactional nature of psycholinguistic processing remains weak. Only a return 
to the original source— 5e/n/o?ic Theory^-oBn permit a cognitive view of social 
interaction with sufficient muscle. 



Semiotic or Broad Pragmatic Theory 

Semiotics, or the study of signs, sign systems and the meaning generated when 
such systems are translated, boasts as its protypical example the study of language 
(linguistics). A prominent feature of semiotic analyses of discourse involves 
examining icons, indexes and symbols in terms of the networks or relationships 
in which they occur (context) and by so doing, arriving at an interpretation of the 
given text. The key to this task of producing meaning from such signs lies in the 
inclusion of the whole system and its complex networking with other systems. 
Charles Sanders Pierce, in his philosophical writings on the sorts of logical 
reasoning man uses to function in his environment, equates semiotics with a term 
he coined. Pragmatics^ or the process of inference generation and interpretation 
(Oiler 1989). Consequently, semantics (at the content level) is designated as the 
organizer of the sign systems, while pragmatics (at the relationship level) is 
assigned to the actual interpretation of such systems by individuals. 

Pierce suggests that only three types of reasoning are available to the human 
mind: the first involves drawing inferences based on experiential, sensory 

perceptions of a physical sort (Abduction-iconic signs); the second type relies on 
probability and experimentation via experience and includes affective and kinesic 
information exchanges (Induction-indexical signs); the third type differs from 
abduction and induction in its lack of reference to time and space— here we have 
the realm of theory and universal rules (Deduction-symbolic signs). The pivotal 
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point of all three reasoning processes is the world of experience and all logical 
conclusions must be verified by this critical feature (Oiler 1991; 1992). 
Experience (social and cognitive) grounds all of our perceptions which are in turn, 
filtered through langxiage; consequently context is the primary ingredient of 
discourse analysis. 

How can this trichotomy account for sociolinguistic competence? The 
socialized, conventional parts of language and the struggle for normative 
conformity and consensus clearly demand inductive reasoning. For example, 
qjpropriate sensitivity to politeness norms is acquired through experimenting and 
hypothesizing; after a few bad experiences with violating these norms, we draw 
inferences tha'i this specific behavior in this specific time and place must not be 
acceptable to members of our cultural community. Another example involves 
gestures and other paralinguistic communicative acts. Gestures "guide us in the 
way we manage and negotiate relations with things, events, and persons in our 
environment" (Oiler 1991, p 5), Thus, the idea that meanings of signed 
expressions are socially recognized because they develop through interpretations 
conveyed in their contexts and circumstances, is in fact a social extension of 
semiotic philosophy (see Halliday & Hassan 1989; Hodge & Kress 1988). By 
infusing a broad pragmatic framework with the face-to-face concerns of human 
interaction (i.e., speech community conventions and face needs), we see how 
comfoitably certain communication models fit within this powerful structure. At 
the same time, we begin to understand how comparatively weak Speech Act 
Theory and Politeness Theory appear when used in isolation. Even if a 
psycholinguistic support mechanism could be arranged, the relationship with other 
sign systeiTis (i.e., kinesic, prosodic) remain disconnected. 

Speech Act Theory 

Speech Act Theory (Austin 1962; Searle 1969) classifies verbal messages 
according to their communicative function. A triad of these basic categories 
include: a locutionary act or the act of saying something in quotable, literal 

terms (he said to "Call back later"); an illocutionary act or the act identified by 
the statement which conveys the force or function of the utterance— the speaker's 
intended meaning (I suggested, urged or ordered them to call back later); and a 
perlocut ionary act or the act performed as a result of the utterance in terms of a 
commentary from the speaker’s perspective— what the hearer understands from the 
utterance (he persuaded us to call back later). 

Specifically, Searle (1975) suggests that the illocutionary force of an utterance 
consists of two parts, namely a proposition and a force or function-indicating 
device (e.g., word order, mood of the verb, punctuation, etc..). So that a 



U 

% ‘ 



170 



Robin 0. Zuskin 



Statement can be converted into a question or an emotional co mman d simply by 
applying one or a combination of these devices. The rationale underlying this 
maneuver is that by altering the force of a message, its interpretation changes. 
This distinction between literal and implied force represents the star feature of 
Speech Act Theory and, some claim, the key to understanding indirect speech 
acts. 

The question still remains as to how the speaker’s intentions are actually 
inferred by the hearer. If reviewed from a semiotic, Piercian perspective this 
psycholinguistic cavity can be filled. Since each participant comes into the speech 
scenario with a set of communal and personal experiences about two sets of 
expected behavior norms— linguistic and cultural— meanings of messages will be 
inferred according to this backdrop. But it must be pointed out that the text itself 
also contributes to this comprehension process. So that a web of indirect 
inferences (Oiler and Jonz, In press, p.23) are drawn from three sources— the 
speaker, the hearer and their relationship with the text itself— although not as 
simply and direct as Searle seems to suggest (Figure 2) . 

direct ftccMi 




FIG. 2: Speech Act Theory via a semiotic exchange of 
information and intentions. 

From Cloz£ and Coherence by J.W. Oiler and J. Jonz (In press) 

b 




Atseuing 12 Sociolioguistic Competeoce: 



171 



Should these muitiKlirectional interpretations lack coherence by virtue of an 
incomplete triad, successful communication cannot occur; an unexpected in- 
formedness or ini^propriateness to the situation at hand, in effect, riders a 
message incomprdiiensible or uninterpretable. Sociocultural differences come into 
play because a common "world of experience" is essential for effective "pragmatic 
mapping" between the facts (what we know to be true) and representations of 
those facts (Oiler 1992). This relationship assures that a sense of logical 
connectedness and consistency escort the interpretation process so that responses 
are relevant and appropriate; it also generates communicative breakdown if 
expectations lack mutuality. Flowerdew (1990) views this lack of coherence as 
reflective of the discrete point discourse analysis approach underlying Speech Act 
Theory. Since it fails to explain why the individual linguistic acts do not 
contribute to the total meaning of the conversation, its utility is questionable. A 
possible ^eason for this disappointment is because illocutionary forces may in fact 
"spread over a number of utterances and that one utterance may share more than 
one force" (Flowerdew, p 94) so that separating speech acts into distinct, isolated 
units does not make sense. By denying this interdependent state of affairs. Speech 
Act Theory can only generate a set of vaguely related functions or tasks such as 
in a fiinctional-notional syllabus. Without some linguistic glue, an incomplete 
relationship persists between verbal acts and highly contextualized social 
interactions. Ironically, despite this and other serious shortcomings (i.e., the 
belief that speech acts can be finitely numbered or that there exists no means for 
explaining nonverbal speech acts), researchers in a wide spectrum of disciplines 
from linguistics to sociology to anthropology to communication have placed their 
trust in Austin and Searle’s philosophical concepts. 

Politeness Theory 

Brown and Levinson’s chapter on politeD-?ss phenomena (1978) constitutes an 
example of the great impact a single publication can have on the linguistic 
academic community. Their notion of "face-threatening acts" (FTAs) needs to be 
examined more closely however. It seems that although it may offer some 
insights into social behavior norms (Tracy 1990), this theoretical model conveys 
a rather incomplete picture of communication via language. 

Brown and Levinson’s iimovative theory stipulates that in order to predict the 
politeness strategy of choice, two criteria must be addressed: (1) whether the 
illocutionary act will be on record (state intentions clearly, e.g., "I promise I’ll 
clean my room") or off record (ambiguous, more indirect statements of intention, 
e.g., "Didn't 1 tell you I would clean my room?", rhetorical questions, metaphor, 
irony, understatement, etc.); and (2) in the latter situation where redressive action 
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is expected, if positive face needs (solidarity) vs. negative face needs (freedom 
from impeded action) require attention. Following, but by no means limited to 
Grice’s (1975) cue. Brown and Levinson (1978 p 64-65) concede that it is "mutual 
knowledge" that drives their strategic model. But where Grice’s Cooperative 
Principle focuses on mutual understanding. Politeness Theory becomes 
coxmterbalanced by certain protective measures in a frame loaded with risk and 
danger. Speech participants need defensive equipment to safely engage in 
conversation. In short. Politeness Theory prioritizes social pressures in explaining 
verbal interaction. These social factors, when combined under certain 
circumstances, determine the specific choice and sequencing of conversational 
strategy. 

Evidence for the Politeness Theory lies in its generalizability across cultures. 
Through much cross-cultural empirical data (e.g., Olshtain & Cohen 1983; Blum- 
Kulka et al. 1989; Kitao 1990; Beebe & Takahashi 1989), it appears that 
politeness may be a universal occurrence. Accordingly, this paradigm has won 
much respect among sociolinguists. Despite the support it has received. Politeness 
Theory cannot account for all dimensions of linguistic functions; some 
communicative acts may not even involve facework such as intrapersonal message 
exchanges. Cognitive processing, particularly from an interlanguage perspective 
(Gass 8l Varonis 1991), also remains a mystery when Politeness Theory is solely 
applied. 

Politeness Theory , in summary, stands on firm groxmd, although it lacks the 
amount of power for which it originally strived. It must be viewed under a 
pragmatic umbrella in order to gather adequate support for carrying out its 
theoretical responsibilities. Because it attempts to forecast linguistic behavior 
from social experiences, it remains pragmatically sound. It helps to explain some 
of the illocutionary force variability, particularly indirect speech acts-but not all 
of it. A gap remains where social needs extend beyond those of face, where 
kinesic or prosodic channels interact with speech or where Western socialization 
objectives may not account for the notion of face cross-culturally (Gu 1990; 
Matsumoto 1988). 

Pragmatic theories are the only ones capable of accounting for the situated 
aspect of language use; things get very messy due to a collage of interacting 
variables like power hierarchies, social distance, gender subcultures, age uid the 
like, PLUS factors like native speaker negotiation/accommodation patterns, L2 
interlanguage development, individual variation (i.e., personality, empathy) and 
others relevant to the translation of semiotic material. A broad version of 
pragmatic theory welcomes these variables since the context itself provides 
sufficient information for the work of making meaning. In other words, by 
highlighting the unique sociocultural baggage each interlocutor brings to the 
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message exchange forum and how it contributes to its interpretation in that 
particular "strip of activity", we can arrive at a more accurate response to 
Coffman's (1974) perceptive question: WicU *s going on in this frame? A semiotic 
perspective actually authorizes us to look at each communicative encounter on a 
case-by-case basis: within a particular situation we find particular interactants 
who are influenced by particular cultures and social communities to convey and 
interpret verbal and nonverbal discourse in particular ways. 



Testing L2 Sociolinguistic Skills 

In the area of L2 measurement, sociolinguistic competence has influenced 
various testing paradigms. Bachman's (1990) popular multi-dimensional 
framework of communicative competence, for example, divides this grand ability 
into two principle skill subcategories: Organizational Competence (grammatical 
competence and textual competence), and Pragmatic Competence (illocutionary 
competence and sociolingusitic competence). Bachman (p 90) defines the latter 
as "the knowledge of the pragmatic conventions for performing acceptable 
language functions and... the knowledge of the sociolinguistic conventions for 
performing language functions appropriately in a given context." Bachman also 
includes a psycholinguistic thread to his model, an innovative addition to many 
pragmatic theories. Yet no explanation about how these components interact with 
one another has been tendered. Can someone be sociolinguistically competent 
without having first reached a certain level of grammatical ability? How much of 
sociolinguistic competence depends on discourse cohesion and organization? 
Moreover, where do alternate communication modalities such as intonation, 
gestures and facial expressions fit in? These are issues that need empirical 
attention before any L2 student is deemed sociolinguistically competent or not. 
These are issues which tests claiming to assess overall communicative competence 
cannot ignore. Shohamy (1984) explains the urgency of developing this type of 
assessment tool: 

Till today, sociolinguistic proficiency has rarely been tested. Most 
language tests still focus on linguistic aspects as the major criteria for 
test construction. Overlooking the state of the art in knowing what a 
language means today, implies the construction of tests which fail to tap 
the construct of language proficiency in its full and complete 
definition... The construction of such tests will involve imposing 
rigorous measurement criteria to convert this sociolinguistic information 
into tests of sociolinguistic proficiency. (p.l61) 
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Validity & Roleplaying: The Discourse Completion Test 

Endeavors to assess communicative competence, or at least the portion of that 
con^pet^ce that language tests have successfully been able to tap (i.e., 
proficiency), have more recently included a sociolinguistic component. As more 
and more L2 curricula begin to include the various ingredi^ts of linguistically 
realized politeness norms (e.g., register variation as dictated by perceived status 
of interlocutors, by s^itivity to situational formality and the face needs of 
interlocutors) as well as other socio-context dependent linguistic phenomena, the 
development of a valid process of evaluation becomes in^)erative. One procedure 
which relies on a roleplay format is currently under analysis both as a linguistic 
research tool and as a candidate for a supplementary position in tests claiming to 
assess overall communicative competence (i.e., an alternative to oral profici^cy 
interviews). 

The Discourse Completion Test (DCT) elicits responses to problematic, 
contextually-specific prompts as participants, in writing or orally, roleplay their 
responses. It is somewhat analogous to a cloze test where the blank is beyond the 
one-word level and instead begs for an extended, pragmatic level response. Both 
the cloze and DCT task parallel one another in terms of tapping inferencing skills 
and sensitivity to coherence of text. 

The origin of this sociolingxiistic instrument (Blum-Kulka 1982) promotes a 
written interaction with often more than one rejoinder between the respondent and 
a hypothetical character in the second person. In other words, the respondent of 
the DCT was originally meant to write a least two separate utterances. The DCT 
has evolved gradually over the past decade into several different modified 
versions, including some using a third person perspective, many requiring only 
a one-utterance written response from subjects, some which allow for oral 
response and, naore recently, an extended interaction nearing an oral interview-like 
character. Oral responses, furthermore, have been videotaped for later assessment 
or rated by trained NS judges immediately upon oral performance (see Cohen & 
Olshtain 1991). All the various DCT adaptations which have been studied share 
a common characteristic: they have all relied on a textual description of a 

particular situation which requires examinees \o act as if by demonstrating their 
skills of empathetic roleplay acting. 

What is roleplaying? From the outset, some clarification of this pedagogical 
strategy is needed. By dissecting this verbal compound, we find that when 
students assume a rolCy they take on a part (either their own or someone else’s) 
in a specific situation. Play means that this role is performed in a safe 
environment so as to promote creativity and motivation while reducing stress and 
anxiety (Ladousse 1987). This tactic has been used in L2 classrooms for quite 
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soBLie time, especially during the audio-lingual period. However, communicative 
teaching principles have converted a controlled, scripted roleplay into a more free, 
inq)rovisational technique. The DCT capitalizes on the proven benefits that this 
methodology has demonstrated in the teaching ar^a (Bardovi-Harlig et al. 1991) 
and applies them to the work of assessment. It is very important to keep in mind 
that despite its popularity among instructors, teaching and testing have different 
criteria because of their different objectives. 

From a semiotic approach, however, context-related revisions of the DCT are 
absolutely necessary. Despite their well-meaning inclusion of situationally-specific 
discourse frc^mes, the schematic information with which DCT researchers have 
supplied examinees does not convey enough about the relationship (e.g., status, 
positional identities) , between the speaker and hearer. Furthermore, the 
unextended featuie of these examples illustrates the unlikelihood of their 
occurrence in natural discourse. Students do not know enough about the 
characters they are role-playing nor about their conversational partners. The 
contextual tapestry is not complete; yet, NNSs are supposed to respond as if it 
were. The use of hypothetical simulations depends on an abundance of schema 
if appropriateness intuitions are to be stimulated. Since we can only infer 
examinee’s underlying sociocultural competence based on their performance in the 
role-play, the script must provide for a more effective conveyance of the 
relationship between the characters. The latest studies on roleplay effectiveness 
as an evaluation tool indicate that communicative performance is highly situation 
specific, so that the context of each scenario can actually affect examinees’ speech 
act strategies (Cohen & Olshtain 1991). Consequently, while social, cultural and 
personal factors influence speech act realization choices among NNS examinees, 
situation outweighs these others. This point should be incorporated into the DCT 
design. 

The following role-play prompt exemplifies the DCT as a data elicitation 
instrument currently used in the Cross-Cultural Speech Act Realization Project 
(CCS ARP). This collaborative study examines the production of NNS requests and 
apologies across 7 linguistic cultures using a strictly third person type of roleplay 
(see Blum-Kulka et al. 1989). The DCT item (1) represents one of the more 
popular DCT formats: : 

At the professor’s office 

A student has borrowed a book from her professor, 

which she promised to return. 

Professor: Ann, I hope you brought the book I lent you. 

Ann: 

Professor: OK, but please remember it next week. 
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Other variations (Beebe et al.) replace the student’s name with "you" in the 
above. Rintell & Mitchell (1989) suggests removing the sandwich approach by 
switching the blank line of dialogue to the end of the critical incident description. 
The teacher-student interaction conveyed above would be modified into a more 
cohesive prompt as in the following example (2): 

Ann, a student at the university, borrowed her professor’s book last 
week. She promised to return it today in class but is not finished using 
it. She would like to borrow the book a few more days. Imagine you 
are Ann. What would you say to your professor to get an extension on 
the book loan? 

YOU: 

Regardless cf the DCT prompt format used, some unavoidable pragmatic 
problems prevail. Despite their well-meaning inclusion of situationally-specific 
discourse frames, the tCvSt’s original authors seem more concerned with contextual 
clues than with contextual ones. The schematic information with which they have 
supplied examinees does not convey enough about the relationship (e.g., status, 
social distance) between the speaker and hearer. Students do not know enough 
about their hypothetical conversational partners to determine the degree of 
imposition of the FTA. The use of hypothetical simulations depends on an 
abundance of schema if appropriateness intuitions are to be stimulated; there 
exists a camouflage of communicative intent which hides a lack of real-life 
applicability. Since we can only infer examinee’s underlying sociolinguistic 
competence based on leir performance in the role-play, the script must provide 
for a more effective conveyance of the relationship between the characters. 
Offering a preliminary introduction of the two basic characters and then retaining 
those two speech participants throughout the 16 different situations might be 
helpful. 

This roleplaying methodology conveys a high degree of face validity (Clark 
1978). Few test developers would question this, although some have doubted the 
realistic nature of using written roleplays for speech that is noimally produced in 
an oral fashion (Cohen & Olshtain 1991). Face validity or authenticity of 
communicative competence is one of the major issues confronting test developers 
today. The appearance of a test can actually affect test performance if examinees 
perceive the test to be measuring what they expect it to measure even though the 
test may not tap the targeted psychological construct. 

As discrete point tests have become a near taboo in the field of language 
testing development (although in practice they remain popular), researchers are 
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determined to produce a more holistic, natural means of evaluation. But does that 
criterion necessarily translate into direct testing? The overlap or area of gray that 
results when attempting to delineate trait from method is problematic due to the 
arbitrary cut-off point we set in our validation work (e.g., when do the roleplay 
prompts cross over from method into the category of trait?); and it may be this 
problem which complicates our interpretation of test statistics (Stevenson 1981). 

Nevertheless, Clark (1978) defends direct tests as advantageous because 
ethnographic observation is simply too impractical as a systematic testing tool. 
Authenticating opportunities for communicative interaction vrithin the limits of 
testing time and facilities means that real life situations are approximated to the 
greatest extent possible. Wesche (1985) supports the need for direct testing since 
predictive validity is almost guaranteed; the inferred skills demonstrated during 
the test can be extended to real life language use with some degree of confidence 
since the test supposedly mirrors such use. But most of these kinds of claims lack 
empirical support, as we see that rationalization does not equal validation. The 
inherent artificiality despite attempts to appear natural stems from a lack of 
puposefulness and contextualization (Raffaldini 1981). For example, when 
communicative functions are targeted as if they occur in isolation from others, 
when prosodic and kinesic cues are awkwardly missing or when the relationships 
between interlocutors are not clarified for examinees, pragmatic mapping is made 
difficult if not impossible. 

With all the inexact cognitive probing test developers must contend with, face 
validity seems, ironically, to provide a way to appease various audiences: 
students, administrators, teachers, etc. The facade is effective only until students 
are misplaced in programs or their problems are misdiagnosed. Face validity can 
be a plus only after true construct validity has been established. Some questions 
affecting the construct validity of roleplays include: Do role plays elicit 
opportunities for examinees to generate evidence of their native-like ability in 
spoken discourse? Do they provide enough context to students so that a 
hypothetical response can be gleaned from students * real experiences ? Or better 
vet, do the discourse prompts promote appropriate and polite responses by 
providing students with culturally-specific discourse signals that native-speakers 
would recognize immediately? If any of these questions cannot be answered with 
conviction, it is perhaps due to the weakness of the theories from which roleplay 
and interview tools gather support. 
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CONCLUSION 



The communicative objectives of the DCT appear quite in line with pragmatic 
principles; yet, because its developers have not specifically admitted that a 
communicative assessment strategy is in fact one based on broad pragmatic theory, 
they can only muster a vague notion of its validity. The DCT could be revised, 
however, in correlation with clear pragmatic criteria. Its estrangement from an 
empirically-supported theory of language— one which is not plagued with criticisms 
(e.g.. Speech Act Theory) or one which can account for more than mere 
sociolinguistic phenomena (e.g. Politeness Theory )-hinders its utility as a tool for 
measuring sociocultural competence. Much work remains in terms of clarifying 
specific construct validity requirements of roleplays, both written and oral. Only 
as more empirical studies produce evidence beyond the anecdotal and intuitive 
level, can we anticipate espousing that various roleplay techniques indeed measure 
what they purport to measure. 

While a written description of a scenario can certainly provide examinees with 
numerous contextual cues, to fully enable them to respond appropriately, we must 
enrich this context with more than mere verbal signs. As Semiotic Theory 
explains, nonverbal cues often carry much of the affective and sociocultural 
message (Oiler 1992; Feldman et al 1991). The inclusion of time, moreover, 
makes expression of signs via video or film "more complete" or more accurate in 
specifying a particular event than most other textual formats (Korac 1988). Thus, 
why not reinforce the text itself with some alternative modes of communication 
such as gesture and prosody? The same critical incident that was originally read 
by participants can then be interpreted in gestures and in sensory -motor modalities 
in addition to pure linguistic representations. This approach considers the 
variation of learning styles and culturally-based inferencing processes among L2 
students. Moreover, if we want to know how well examinees can use the 
elements of language in real-life communication, we must offer them a testing 
forum which not only simulates real-life communication, but which provides 
meaningful contexts where interaction with interlocutors or with other examinees 
enLinces successful outcomes. 



Future Research 

For this reason, a video version of the DCT has been produced by this writer 
which conveys each situation audio-visually. Before each 10-30 second critical 
incident appears on the video monitor, a brief preamble is provided. Examinees 
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can read and listen to the information regarding character roles and relationships 
as they are "set up" (i.e.> giv^ a communicative objective) for the elicitation of 
one of four speech acts: apology, request, refusal or conq)laint. They are finally 
asked to respond as if they were actually in the particular context of the character 
they see and hear on the screen. Responses can be videotaped or, if time and 
money constraints do not permit, written questionnaires used in the traditional 
DCT can be substituted. Research on this and other methodological variations of 
the DCT are in demand before we can confidently suggest its adoption for use 
(e.g., Rintell & Mitchell 1989; Cohen & Olshtain 1991; Cohen in press). 

In the L2 pedagogical niche known as communicative testing, the rationale for 
revising our theoretical model is twofold: first, we absolutely need to establish 
construct validity to justify psycholinguistic measurement instruments; second, we 
need to collectively work toward understanding h uman co mmuni cation so that 
refinement of linguistic theories can follow. In our eflforts to link theory to L2 
teachi n g and testing practices, we must work toward validation of our pragmatic 
framework—a model which accounts for natural, non-ideal discourse in a variety 
of sociocultural contexts. In so doing, we need to recycle some good ideas, throw 
out those lacking support and reorganize for effectiveness of explanation. 
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